A Integrated Suite for Database Benchmarks

An Integrated Benchmark Suite for Database Systems

This paper is a preliminary description on an integrated benchmark suite for database systems, SIMS, which offers generic benchmarks and custom benchmarks to users in a unified interface. Currently SIMS supports three generic benchmarks (Wisconsin, BORD, and TPC-C) and four user-defined benchmarks. Any new custom benchmarks can be easily accommodated into SIMS. SIMS consists of four modules. Ea...

PUMA: Purdue MapReduce Benchmarks Suite

A tool suite for creating question answering benchmarks

We introduce the BIOASQ suite, a set of open-source Web tools for the creation, assessment and community-driven improvement of question answering benchmarks. The suite comprises three main tools: (1) the annotation tool supports the creation of benchmarks per se. In particular, this tool allows a team of experts to create questions and answers as well as to annotate the latter with documents, d...

Database Benchmarks

Performance measurement tools are very important, both for designers and users of Database Management Systems (DBMSs). Performance evaluation is useful to designers to determine elements of architecture, and, more generally, to validate or refute hypotheses regarding the actual behavior of a DBMS. Thus, performance evaluation is an essential component in the development process of well-designed...

A Workload Generator for Database System Benchmarks

There are cases in which database system benchmarks should be executed in a realistic environment to get meaningful performance results. We propose a workload generator that helps benchmarks be executed in an environment similar to the real world in terms of resource status. The workload generator can create memory-bound, CPU-bound, and I/O-bound workloads. The workload generator allows users t...